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ABSTRACT 


Auto tracking mobile robot is a device that able to detect and track a target. 
For an auto tracking device, the most crucial part of the system is the object 
identification and tracking of the moving targets. In order to improve the 
accuracy of identification of object in different illumination and background 
conditions, the implementation of HSI color model is used in image processing 


algorithm. In this project HSI-based color enhancement algorithm were used 

for object identification. This is because HSI parameter are more stable in 
Keywords: different light and background conditions, so it is selected as the main 
parameters of this system. Pixy CMUcam5S is used as the vision sensor while 
Arduino Uno as the main microcontroller that controls all the input and output 
of the device. Moreover, two servo motors were used to control the pan-tilt 
movement of the vision sensor. Experimental results demonstrate that when 
HSI color-based filtering algorithm is applied to visual tracking it improves 
the accuracy and stability of tracking under the condition of varying 
brightness, or even in the low-light-level environment. Besides that, this 
algorithm also prevents tracking loss due to object color appears in the 
background. 
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1, INTRODUCTION 

The contribution of robots is rapidly increasing day by day as robots are starting to substitute humans 
in everyday tasks. At the same time robots can do tasks that would either be impossible for human being to do 
or it would take a longer time for him or her to complete it. Thus, robots make human’s work easier and more 
productive. The function of robot can be further increased by giving vision to the robot. Color vision based 
mobile robot path tracking is proposed by Luo et al. [1]. In this proposed system, path following algorithm is 
proposed based on the data extracted from HSI (Hue, Saturation, intensity) color model using fuzzy control. 
As the image 1s captured, it will be converted into HSI color model and using segmentation method, the path 
is extracted from the image using optimal threshold in HSI module. The path skeleton is then extracted using 
skeleton extraction method. In addition, fuzzy control is used for robot path tracking control. 

Soans et al. [2] used adaptive color threshold method which is equipped within a mobile robot to 
detect and follow a particular color of an object. As the robot equipped with mechanical arm, the mobile robot 
is able to pick the target object. Based on the proposed method, the captured images are transferred to a color 
thresholding algorithm to detect the target. The noises are filtered out and finally the colored object is detected 
and picked up. In addition, as mentioned by Soans et al. [2], one of the main problem in color tracking is the 
insignificant of current color thresholding technique which caused by e.g. reflection of smooth ground plane. 
This reflection results in false object detection and consequences leads to false color tracking. On the other 
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hand, the variation of surrounding lighting conditions also leads to false color detection and tracking. These 
problems are the focus objectives in this paper. 

People detection and tracking system based on real-time RGB-D for mobile robot is proposed by 
Fang et. al. [3]. In the proposed system, an open source robot operating system (ROS) is implemented to a 
mobile robot to track a target. In the system, the feature of the target is extracted before the depth information 
is collected and used to track the target based on the nearest point position information. Then, the 
implementation of CAM-Shift algorithm which is based on RGB information is applied to improve the anti- 
interference ability. 

For the purpose of image contrast improvement of inhomogeneous illumination, Abdul Ghani and 
Mat Isa [4] have applying recursive adaptive histogram modification which focuses on applying clip-limit and 
gray-level mapping of the captured images. From the results, the output images show the significant output by 
producing a homogeneous illumination of the images. This method could reduce the noise level in the captured 
image of the auto-tracking mobile robot. In addition to this algorithm, unsupervised contrast correction through 
integrated-intensity stretched-Rayleigh Histogram [4], could become another option to address low contrast 
and non-homogeneous illumination images. 

This auto tracking device can be used to enhance the surveillance system. For instance, it can be used 
to track valuable things. For example, this system can act as a moving CCTV as it not only able to observe but 
it also able to track things. Besides that, new technology invents an electric wheelchair where the user can 
control the movement of wheelchair. Now, with auto tracking technology, the electric wheelchair can be 
convert into an automatic wheelchair whereas the movement of the wheelchair to a certain point is done 
automatically based on line or moving object’s color. In addition, the technology also can be applied to a 
luggage or shopping trolley where the system will automatically track and follow its owner. 

In this project, the camera will be attached at the mobile robot as a component to move from a location 
to another location based on the moving target. Microcontroller with image processing implementation plays 
an important role. A color-based filtering algorithm is used for object tracking. Color-based filtering methods 
are popular because they are fast, efficient, and relatively robust. 

Next, a microcontroller is also an essential part in auto tracking device. Microcontroller act as the 
brain of the device. Once the image has been captured by the camera, the image will be decomposed into its 
individual color channel. Based on these color channels, the total intensity and the maximum intensity values 
of each color channel are calculated. The maximum intensity values between these color channels will 
determine the dominant color of the object for the purpose of the device or mobile robot to identify it. Besides 
that, the microcontroller also sends electrical signal to the mobile robot’s actuators to react to the motion of 
tracked object. 

The direction of this project is to fabricate a simple an auto tracking mobile robot that is able to detect 
an object and tracks it. The main focus of this research project is to improve the detection and tracking abilities 
of the target color, as mentioned by Soans et al. [2]. The algorithm of image processing to detect the object is 
based on the color of the object. A (Hue, Saturation and Intensity) HSI color-based filtering algorithm were 
used in this project to detect the target. Firstly, the camera will capture the image and the microcontroller will 
process the information and monitor the behavior of the object. Next, when the object starts to move, this 
mobile robot also will follow and tracks the object. At the same time, the mobile robot always maintain distance 
with the object to prevent from collision with the object. Besides that, the vision of the robot is up to 180° in 
X-axis and y-axis as the camera 1s attached to a pan-tilt which is able to rotate. The rotation of the camera is 
control by two servo motors whereas one servo motor is used to control x-axis movement and another servo 
motor is used for y-axis movement. 


2. RESEARCH MATERIALS AND METHODOLOGY 

From the literature review, suitable components for the project were selected. The circuit design of 
the project is developed. This followed by software development of the project. After this, the system is tested 
until the system works according to the objective of this project. Figure 1 shows the methodology flowchart of 
the project of auto-tracking mobile robot. 
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Figure 1. Methodology flowchart of auto-tracking mobile robot 


a. Pixy CMUcam5 


The vision sensor selected for this project is Pixy CMUcam5. The Pixy CMUcam5 vision sensor is a 
fast image sensor that tracks object and it can directly connect to Arduino Uno through ICSP port on Arduino 
Uno board. Besides that, Pixy has its own powerful processor to process the image. Since Pixy has its own 
processor, it will process the captured images from the sensor and extract the useful information. Besides that, 
Pixy come with a color algorithm to detect object’s color. Normally, RGB (red, green, and blue) used to 
represent colors. But, Pixy calculates the hue color and saturation of each RGB pixel from the image sensor 
and uses these as the primary filtering parameters. Thus, conversion algorithm to convert RGB to HSI color 
based is not required in the programming part as the algorithm is already integrated in the Pixy CMUcam5 
image sensor module. Nevertheless, Pixy processes an entire 640x400 image frame every 1/50th of a second. 
This means the camera detected objects’ positions every 20 milliseconds. 


b. Arduino Uno 


In this project of developing an auto-tracking device, Arduino Uno was selected as the controller of 
the system. The Arduino Uno board is a microcontroller based on ATmega328. It has 14 digital input/output 
pins in which 6 can be used as PWM outputs, a 16 MHz ceramic resonator, an [CSP header, a USB connection, 
6 analog inputs, a power jack and a reset button. The camera can be directly connected to ICSP header. 


c. L293D Motor Driver 

L293D motor driver is an integrated circuit chip which is usually used to control. Motor driver act as 
an interface between Arduino and the motors. These [Cs are designed to control two DC motors simultaneously. 
L293D consist of two H-bridge. H-bridge is the simplest circuit for controlling a low current rated motor. 


d. Circuit Design 


The main hardware components selected to build the prototype for auto-tracking mobile robot are the 
Arduino Uno, Pixy CMUcam5 camera, Arduino Motor Shield, servo motor and DC motor. The development 
of circuit design for this project was started with a block diagram as shown in Figure 2. A block diagram is a 


diagram of a system in which the principal parts are represented by blocks and connected by lines to show the 
inputs and outputs of a system. 
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The complete schematic circuit diagram of auto-tracking mobile robot device is shown in Figure 3. 
The main component of the circuit is Arduino Uno as it receives data about the position of tracked object from 
the Pixy CMUcam5. Then, Arduino will control the movement of DC motor according to the motion of tracked 
object. Besides that, the servo motors are directly controlled by Pixy CMUcamS itself. 


Servo Servo 
Motor Motor 
Arduino L293D —->| DC Motor 
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Figure 2. Block diagram of auto-tracking mobile robot 
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Figure 3. Schematic circuit diagram of auto-tracking mobile robot 





e. Completed hardware design 

The following Figure 4 shows the complete mobile robot equipped with Pixy camera for the purpose 
of color detection and tracking. In addition to the camera, the mobile robot is also equipeed with two DC 
motors, motor driver, and pan-tilt mechanism which consists of two inter-connected servo motors. 
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Figure 4. Final hardware design 
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f. Programming flowchart 


Figure 5 shows the programming flowchart of auto-tracking mobile robot. Firstly, when the device is 
turn on, the system will be initialized. After that, the Pixy CMUcam5 vision sensor will find the signature of 
the tracking object. Once the object is detected the vision sensor will calculate the area and x-coordinate of the 
tracking object for every 20 milliseconds. To make sure the tracked object always parallel with the vision 
sensor an optimum area and x-coordinate are set as reference value where the reference values are defined as 
Amax: Amin Xmax ANd Xmin - Tf Areagpject > ATCAmgx means the object is too near to the device. Thus, the 
device will move backwards to keep the device and object at optimum range. Same goes if ATedgpject < 
Are€Qmin , this means the object this far from the device. Thus, the mobile robot will move forward until it is 
at an optimum range. Besides that, x-coordinates are used to determine the movement of the object in x- 
direction. If the object Xop ject < Xmin means the object is at left. Thus, the program will set the right motor 
on and left motor off so that the mobile robot can turn left. Same goes for Xop ject > Xmax condition where for 
this condition the object is to the right. 
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Figure 5. Programming flowchart of auto-tracking mobile robot 


3. RESULTS AND DISCUSSION 

The auto-tracking mobile robot able to tracking objects effectively as proposed at normal condition. 
Several experiments were carried out on the proposed system to ensure the objective of the project is achieved. 
First experiment was to determine the maximum target locking distance. Another objective of this project to 
evaluate the accuracy of colour identification in different illumination and background conditions, and thus 
improves it efficiency in tracking within various environment including low contrast and almost similar 
background-target color environment. Thus, the second experiment is carried out to determine the effect of 
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background colour during object tracking. Third experiment was to exermine the effect of illumination on 
detection and target tracking. 


a. Maximum target locking distance 


The objective of this experiment is to determine the maximum distance of target from the device that 
gives accuracy and efficient tracking. The result of the experiment is in Table 1. Based on Table | it is found 
that when the detection range is increased as the stability of the target locking decreases. At signature range of 
8.5, the vision sensor able to detect target that 240cm away from the vision sensor. However, at this range of 
detection the tracking loss occurs frequently due to unstable target locking as shown in Figure 8. At signature 
range of 1.5 the target locking is very stable as shown in Figure 6. But, the range of detection is very short. The 
most maximum target locking distance and the most stable target lock is given at signature range of 5.5 as 
shown in Figure 7 and this is the optimum setting of signature range for best detection and effective tracking. 


Table 1. Result of Experiment A 


Signature range Range of Detection(cm) Target Lock 

Stability 

1.5 90 High 

2) 125 High 

35 150 High 

4.5 170 High 

a 190 High 

6.5 210 Moderate 

oe) 225 Moderate 

8.5 240 Low 





Figure 6. Pixy output at singature Figure 7. Pixy output at singature Figure 8. Pixy output at singature 
range of 1.5 range of 5.5 range of 8.5 


b. Effect of background during object tracking 


The objective of this experiment was to determine the capability of vision sensor to differentiate target 
and background if both have almost similar color. Besides that, the accuracy of detection and tracking in this 
condition is studied. In this experiment a green ball is used as a tracking object and the background colour also 
set as green colour. This experiment is conducted by manipulating the signature range parameter. The result of 
the experiment is tabulated in Table 2. 


Table 2. Result of Experiment B 


Signature range Noise Level Target Lock 

Stability 

1.5 None High 

2 None High 

Fe None High 

4.5 Low High 

2 Moderate Moderate 

6.5 High Low 

5) High Low 

8.5 High Low 
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Figure 9. Pixy output at signature Figure 10. Pixy output at Figure 11. Pixy output at 
range of 3.5 signature range of 5.5 signature range of 8.5 


Based on Table 2 the best setting for signature range parameter is at 3.5. This is because at this value 
the PixyCMUcam5 gives the best detection and target locking without any noise. At the same time, the green 
background also did not interrupt the tracking as shown in Figure 10. Besides that, it also gives the maximum 
range of detection without noise which is about 150 cm. This means the auto-tracking mobile robot able to 
track an object from a distance of 150 cm.At signature range of 5.5 there is very slight background interruption 
as shown Figure 10. Besides that , at signature range of 8.5 the Pixy CMUcam5 unable to differentiate the 
background and target when both are in same color as shown in Figure 11 , this will cause tracking loss due 
misinterpetion between target and object.Thus, signature range of 3.5 gives the best tracking when the object 
color appear in background. 


c. Effect of illumination during object tracking 


The surrounding brightness also plays an important role in performance and sensitivity of Pixy 
CMUcam5S vision sensor. Firstly, the auto-tracking mobile was tested in various lighting condition. It 1s found 
that the device loss tracking when there is dramatic change in lighting condition. However, the tracking was 
still stable when there is only slight change in illumination. Thus, an experiment was conducted at five different 
surrounding lighting conditions. At the same time PixyCMUcam5 camera’s brightness kept constant. The 
result of the experiment tabulated in Table 3. At this point, we aware with the improvement proposed by Abdul 
Ghani (2018) and Abdul Ghani and Mat Isa (2015) for the enhancement of image contrast. This method will 
be implemented in our next enhancement system for a better color detection with various illuminations. 


Table 3. Result of Experiment C (before calibrating) 
Surrounding Brightness Pixy Brightness Noise Level Target Lock Stability 


Very bright 80 High None 
Bright 80 Moderate Low 
Normal 80 None High 
Less Bright 80 Less High 
Dark 80 Moderate Low 


Based on Table 2 the factors that differentiate all this condition are the stability of the target lock by 
Pixy vision sensor and the noise generated from the background of the tracking object. However, this problem 
can be overcome by calibrating the brightness of Pixy CMUcam5 vision sensor until the target lock is stable. 
Hence, the Pixy CMUcam5 vision sensor must be calibrated every time the mobile robot experience significant 
surrounding brightness change. Thus, the experiment is repeated by adjusting the brightness of Pixy camera 
until the target lock 1s stable and the noise level is reduced. The result is tabulated in Table 4. 


Table 4. Result of Experiment C (after calibrating) 


Surrounding Brightness Pixy Brightness Noise Level Target Lock Stability 
Very bright 20 Less Moderate 

Bright 50 None High 

Normal 80 None High 

Less Bright 100 None High 

Dim 120 None Moderate 
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Table 3 shows the data collected after calibrating the Pixy brightness with the surrounding brightness. 
Results proves that the PixyCMUcam5 able to detect and track object at various illumination. This will make 
the auto-tracking mobile robot ineffective as it will loss tracking when the brightness changes dramatically 
until it is calibrated again. The results of target locking and noise level are shown in Figures 12-15. 





Figure 12. Pixy output at very bright condition. Figure 13 Pixy output at bright condition. (before 
(before and after calibrating) and after calibrating) 





Figure 14. Pixy output at normal and less bright Figure15. Pixy output at dark condition. (before and 
condition. (before and after calibrating) after calibrating) 


4. CONCLUSION 

The implementation of enhancement integrated with filtering methods in auto-tracking mobile robot 
gives a Satisfactory solution to the problems that discussed in the problem statement. To improve the accuracy 
of identification of object color in different illumination and background conditions, the HSI color model is 
used in image processing algorithm. The enhancement and filtering processes in HSI color model has 
successfully integrated to solve the problems of loss of color tracking. Experimental results demonstrate that 
when HSI color-based filtering algorithm is applied to visual tracking it improves the accuracy and stability of 
color tracking under the condition of varying brightness, or even in the low-contrast environment, as discussed 
in the results. Besides that, this algorithm also prevents tracking loss due to similar background color. 

Although the tracking loss issue due sudden lighting change can be solved by calibrating the vision 
sensor. One of the future work that can be done this project is to make high accuracy color detection of vision 
sensor especially when the brightness changes. Besides that, this problem also can be solved by replacing 
PixyCMUcam5 with a more reliable camera that able to adjust camera brightness when the surrounding 
illumination varies. For future improvement, the investigation on the reliable and robust algorithm for better 
color tracking will be included. The robust color tracking will be based on the current finding which are related 
to the problems of inhomogeneous illumination, various background color, and similarity of target and 
background color. 
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